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Abstract 



We show that evolution in a population of adaptive agents, repeatedly 
competing for a limited resource, can come to an abrupt halt. This transition 
from evolutionary to non-evolutionary behavior arises as the global resource 
level is changed, and is reminiscent of a phase transition to a frozen state. Its 
origin lies in the inductive decision- making of the agents, the limited global 
information that they possess and the dynamical feedback inherent in the 
system. 
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An evolving population in which individual members ('agents') adapt their behavior ac- 
cording to past experience while repeatedly competing for some limited global resource, is of 
great interest to the social, economic and biological sciences Arthur || has introduced 

a simple example, called the bar- attendance problem, which typifies the complexity often 
encountered in these different disciplines 0: potential customers of a particular bar with 
a limited seating capacity have to repeatedly decide whether or not to attend on a given 
night each week 0. A special limiting case of this problem, the so-called minority game 
H, is currently attracting much attention [p|-[TT|] and has been shown to have a fascinating 



connection with disordered spin systems ||. 

In this paper we introduce and study a simple implementation of the general bar-problem 
which includes evolutionary learning by the attendees. We find that the evolutionary process 
can come to an abrupt halt as the global resource level is varied. This transition is dynamical 
in origin: it results from the inductive decision-making of the agents, the limited global 
information that they possess, and the dynamical feedback present in the system's memory. 

Consider N agents repeatedly deciding whether to attend a bar with a seating capacity 
(i.e. global resource level) of L. Let the actual attendance at the bar at time-step t be A t . If 
At < L , the outcome is the signal 'under crowded'. In contrast, if A t > L then the outcome 
is the signal 'overcrowded'. Hence, the outcome can be represented by a binary string of 
Os (representing, say, 'under crowded') and Is (representing 'overcrowded'). The outcome 
is the only information made known to all agents: the value L is not announced, and the 
agents cannot communicate with each other. All agents have the same level of capability: 
specifically, each agent has access to a common register or 'memory' containing the outcomes 
from the most recent occurrences of all 2 m possible bit-strings {a} of length m. This register 
can hence be written at timestep t as a 2 m -dimensional vector h t with binary components 

e {0, 1} corresponding to the outcome of the most recent occurrence of history a: we will 
call h t the 'predicted trend' at time t. We assign each agent a single strategy p. Following 
a given m-bit sequence (3, p is the probability that the agent will choose the same outcome 
/if as that stored in the common register, i.e. she follows the predicted trend; 1 — p is the 



probability she will choose the opposite, i.e. reject the predicted trend. 

The 'good' decisions are attending (not attending) the bar with the outcome being 'un- 
dercrowded' ('overcrowded'). The 'bad' decisions are attending (not attending) the bar with 
the outcome being 'overcrowded' ('undercrowded'). After the outcome at a given time t is 
announced, the agents making 'good' ('bad') decisions gain (lose) one point. If an agent's 
score falls below a value d < 0, then her strategy is modified, i.e. the agent gets a new p 
value which is chosen with an equal probability from a range of values, centered on the old 
p, with a width equal to R. Upon strategy modification, the agent's score is reset to zero. 
Hence d is the net number of times an agent is willing to be wrong before modifying her 
strategy. Although this evolutionary procedure provides a fairly crude 'learning' rule as far 
as machines are concerned ||12| , in our experience it is not too dissimilar from the way that 



humans actually behave in practice. Changing R allows the way in which the agents learn 
to be varied. For R = 0, the strategies will never change (though the memory register will). 
If R = 2, the strategies before and after modification are uncorrelated. For small R, the 
new p value is close to the old one. Our results are insensitive to the particular choice of 
boundary conditions employed. For L = (N — l)/2, the model reduces to the evolutionary 
minority game introduced in Ref. |T3 . 



Figure 1 shows the numerical results for the mean and standard deviation of the bar 
attendance as a function of the global resource level (i.e. seating capacity) defined as the 
percentage I = (100 x L/N) %. Here N = 1001, m = 3, d = -4 and R = 2 although the 
same general features arise for other parameter values (see Ref. |14||). Averages are taken 
over 10 4 timesteps within a given run, and then over 10 separate runs with random initial 
conditions: this average is denoted by (•••). Here we will focus on the mean attendance 
(A); the standard deviation of the attendance AA given by [(A 2 ) — (A) 2 ]^; the average 



predicted trend given by (h a ); the average strategy (p) given by f pP(p)dp where P(p) 
is the strategy distribution among the agents. P(p) satisfies Jq 1 P(p)dp = N and is time- 
and run-averaged as discussed above. From the definition of the game, games with cutoff 
V = N — L are related to games with cutoff L in that the mean attendance in the game 



with V can be found from the mean population of agents not attending the bar in a game 
with L. Hence we focus on L > N/2. The resulting symmetry about £ = 50% is clearly 
shown in Fig. 1. 

At £ = 50%, the mean attendance (A) = N/2 while A A is smaller than for the 'random' 
game in which agents independently decide by tossing a coin (AA random ^/N /2 = 15.8). 
As L and hence £ increase (£ > 50%), the mean attendance (A) initially shows a small 
plateau-like structure, then increases steadily while always lying below the value of L. This 
increase in (A) occurs despite the fact that the value L (and hence £) is unknown to the 
agents. The standard deviation AA increases rapidly despite the fact that the number 
of available seats in the bar is actually increasing. It seems that by increasing the level 
of available resources, the system actually becomes less efficient in accessing this resource. 
Most surprising, however, is the abrupt transition in both the mean attendance and standard 
deviation which occurs around £ c = 75%. (The precise value of £ c depends on the strategy 



modification range R [0]). For £ > £ c , both the mean attendance and standard deviation 
become constant, regardless of the increase in the seating capacity L. In this region, the 
bar is practically always undercrowded since ((A) + AA) <C L, thereby offering a significant 
'arbitrage' opportunity for the selling of 'guaranteed' seating in advance. 

Figure 2 shows the corresponding variation in the average predicted trend (h a ) and the 
average strategy (p), as a function of the global resource level I. Remarkably, (h a ) has a 
step-like structure with abrupt changes at £ c and £ = 50%. These abrupt transitions are a 
direct result of the dynamical feedback and memory in the system. A random h t generator as 
provided by an exogenous source - e.g. sun-spot activity for prediction of financial markets - 
will not contain these discrete steps and hence does not reproduce the corresponding abrupt 
transitions shown in Figs. 1 and 2. For example a coin-toss would yield (h a ) = 1/2 for all 
l\ however the endogenously-produced predicted trend in the present model only takes this 
value at exactly £ = 50%. 

The strategy distribution P(p) is symmetric at £ = 50%, with peaks around p = and 
p = 1 [[0| [11|. As £ increases, P(p) becomes asymmetric. The peak around p = 1 becomes 



larger while that around p = becomes smaller: agents hence tend to follow the predicted 
trend more often than not. However, an abrupt change occurs at £ = £ c . Figure 3 (upper) 
shows P{p) just either side of £ c , at £ = £ c ± 2%. For £ = £ c — 2% (solid line) the asymmetric 
distribution is clear but P(p) remains non-zero for all p. For £ = £ c + 2% (dashed line), 
Pip) = f° r P < 1/2 but has an irregular shape for p > 1/2. The final form P(p) depends 
on the initial strategy distributions at time t = and the strategy modification range R. 
For £ c < £ < 100%, the distribution P(p) and hence (p) remain essentially unchanged. 

Figure 4 shows the average lifespan per agent defined as r = Jq P(p)A(p)dp, where A(p) 
is the average number of turns between strategy-modifications experienced by an agent at p. 
As £ —>■ £ c , this lifetime r becomes effectively infinite (i.e. the duration of the simulation) and 
remains at this value for all £ > £ c . Above £ c all agents have a strategy value p > 1/2: the 
agents do not modify their strategies and hence keep this same p value indefinitely. There 
is hence no evolution in this system for £ > £ c , despite the fact that the bar has increased 
seating capacity which could be exploited (Fig. 1). In essence, the agents are winning 
enough points to avoid strategy-modification: although the population as a whole is not 
acting optimally, the system chooses not to employ any additional evolution. The frozen, 
non-optimal steady state has a huge degeneracy. These findings suggest to us the following 
informal analogy with a magnetic system with < p < 1/2 representing spin-down and 
1/2 < p < 1 representing spin- up. The point £ = 50% corresponds to an antiferromagnet 
at zero field: since P(p) is symmetric, there is no net magnetization. As the external 
'field' (£ — 50)% increases, there is some weak paramagnetism yielding a small value of the 
magnetization because P{p) is now biased towards p = 1 (Fig. 3 upper graph, solid line). At 
the critical field (£ c — 50)%, there is a transition to a ferromagnetic state (i.e. pure spin-up 
with p > 1/2) as a result of the change in P(p) shown in Fig. 3. 

A connection between the behaviors of the 'macroscopic' variables in Fig. 1 and the 



'microscopic' variables in Fig. 2, can be made analytically. It can be shown [[[J] that the 
mean attendance 



5 



(A)^N[(h a ) + (p)-2(h a )(p)} 



(1) 



and the standard deviation 



AA&2N[(h a )(l- (h a ))]*\(p) 



1 



(2) 



2 



Here (h a ) and (p) are implicitly functions of t. At £ = 50%, (h a ) = 1/2 and (p) = 1/2 
(Fig. 2) hence (A) « iV/2 in agreement with Fig. 1. For 50% < £ < 4, « 1/4: 

the ratio of the slopes of (A) and iV(p) in Figs. 1 and 2 is 0.50 which agrees exactly with 
the analytic result obtained by differentiating Eq. (1) with respect to (p). Similarly, the 
ratio of the slopes of AA and N(p) in Figs. 1 and 2 is 0.87 which agrees exactly with the 
analytic result obtained by differentiating Eq. (2) with respect to (p). For £ > £ c , (h a ) = 
hence Eqs. (1) and (2) yield (A) « N(p) and A A rs 0, which is again in good agreement 
with the numerical results of Figs. 1 and 2. Note that the result (h a ) = implies that 
the bar is always predicted to be undercrowded: clearly this is consistent with the outcome 
since the mean (A) is well below L and the standard deviation is very small. Hence we 
can see why the undercrowded state provides an attractor for the game dynamics above the 
freezing transition at £ c . We note in passing that the evolutionary freezing which arises for 
our generalized bar model (Figs. 1-4) is qualitatively different from the freezing described 
by Challet and Marsili for the (non-evolutionary) minority game |§. 

In summary, we have reported transitions involving an evolving population of adap- 
tive agents who repeatedly compete for a limited, but adjustable, global resource. Abrupt 
changes arise for both microscopic and macroscopic variables of the system, as the level of 
available resources is varied. The present results depend crucially for their existence on the 
dynamical feedback and non-local time correlations present in the system. 
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Figure Captions 

Figure 1: Numerical results for the mean bar attendance {A} and standard deviation A A 
as a function of the global resource level (i.e. seating capacity) defined as the percentage 
i = (100 x L/N) %. The parameters (see text) are N = 1001, m = 3, d = -4 and R = 2. 
Figure 2: Numerical results for the average predicted trend (h a ) and the average strategy 
(jp) as a function of the global resource level t. 

Figure 3: Numerical results for the strategy distribution P{p) (upper graph) and A(p) (lower 
graph - see text for definition) near the transition point i = £ c . Solid line: £ = i c — 2%. 
Dashed line: I = l c + 2%. 

Figure 4: Numerical results for the average lifespan per agent r as a function of the global 
resource level I. 



9 



